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ABSTRACT 



In the current environment of global economic restructuring, 
changes in work and employment, and large-scale immigration to the United 
States, adult basic education (ABE) is facing demands that it be accountable 
for its performance by demonstrating its success in terms of student and 
societal outcomes. The increasing calls for ABE to develop a system of 
performance accountability stem from the following factors and problems: 
regarding what it means to be literate is changing; the purpose of literacy 
remains undefined; ABE'S many stakeholders are not mutually accountable; ABE 
is struggling to create a national accountability system without a national 
service delivery system; and existing tools for measuring learning are not up 
to the task. Thanks to several new initiatives for addressing performance, 
the problems of accountability in ABE are beginning to be addressed at the 
state and national levels. The following principles have been proposed as a 
framework for action that will enable ABE to perform effectively and be held 
accountable for future performance: (1) agree on performance; (2) build 

mutual accountability relationships; (3) develop capacity to perform and be 
accountable; and (4) create new tools to measure performance. (109 
references) (MN) 
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EXECUTIVE SUMMARY 

Adult basic education (ABE) has long been viewed by many educators 
and policymakers as a tool for addressing social and economic problems. 
Now, in a context of global economic restructuring, changes in work and 
employment, and the largest immigration to the USA since the early 
1900s, ABE must demonstrate its success in terms of student and societal 
outcomes. In short, ABE is facing demands to be accountable for its 
performance. 

A focus on results is new for adult education, and potentially transforming. 
Performance accountability offers the chance to re-focus on what 
learners accomplish by participating in ABE and to re-orient every aspect 
of the system to achieve the best results. 

It is tempting to rush into developing measurement and reporting systems. 
Experiences with performance accountability show that this would be a 
mistake. Agreement on what to measure must be established first. 

This policy paper lays out key issues in performance accountability and 
presents recommendations for policy and action. It is based on the 
literature from education, government, management, and other fields, and 
draws on interviews with researchers and adult education leaders at state 
and national levels. Its recommendations were informed by a discussion 
with a group of experienced adult education practitioners and researchers. 

Why change? What is the problem? 

Past policy is no longer a guide for action. The 1966 Adult Education 
Act, which has governed ABE for more than 30 years, provides little 
guidance in developing performance accountability. The act charges 
recipients of funding with enabling adults to "acquire the basic educational 
skills necessary for literate functioning." Definitions of literate functioning 
vary and are hotly debated. Over time, views of what literacy means have 
shifted from academic skills such as decoding text, to functional skills, 
such as being able to perform certain tasks using literacy skills. Recent 
research has changed radically our understanding of literacy. Literacy is 
now described as multiple "literacies" rooted in particular social contexts. 
This change in definition shifts the focus even further from abstract skills 
to real-life practices. 
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This conceptual change requires a re-vision of what "performance" 
means. When literacy meant mastery of what was taught in schools, 
performance was testable. When literacy shifted to the notion of 
functional competency - being able to perform certain tasks using literacy 
skills _ the issue of performance became more complex. Tests had to 
identify which real-life literacy tasks should be included and which not, 
with no theory to guide which to choose or how to create scales of 
difficulty. Test developers had to assume that performance on the test 
equated with how well the student performed the real-life equivalent task. 
The recent research on literacy in its social context has been carried out 
through careful observations of literacy events and activities which shed 
light on prevailing literacy practices. While it shifts the focus to 
performance in life, not in test situations, this new research has not yet 
been incorporated into practice, assessment, or policy. 

Agreement on what "literate functioning" means is crucial to 
accountability. Is literacy a right or a necessity for the good life? Should 
performance be demonstrated in terms of literacy skills - the earlier view - 
or literacy practices, the current thinking? 

The purpose of literacy is not defined. The Adult Education Act of 
1966 had broad social intent. In addition to not strictly defining what 
literacy was, it did not define the purpose of literacy. If a performance 
accountability system is to measure only literacy gain, the task of 
developing such a system would be challenging enough. If a system is to 
measure the achievement of literacy for a predefined purpose, a lack of 
clear objectives makes accountability systems even more difficult to 
develop. The debate in this area centers around whether the purpose of 
literacy education is individual advancement or community development. 
Are the desired outcomes productive workers, good citizens, or merely 
more literate people? Once these questions are answered, the next set of 
questions arise: who defines whether these outcomes are achieved, and 
how much responsibility should the literacy program be asked to take for 
these outcomes? 

Stakeholders are not mutually accountable. Another area of concern 
lies in the mutual responsibility for adult basic education. Many possible 
stakeholders - learners and teachers, administrators and policymakers, 
funders, employers, public school personnel, and taxpayers - may be said 
to have a legitimate concern with the outcomes of adult literacy education. 
All stakeholders are not, however, equal in terms of access to information 
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or ability and power to hold the adult education system accountable. 
Learners, for example, often have limited information and little power to 
change the system. Congressional representatives stand for taxpayers in 
exercising accountability over the adult education system which is 
supported by public money. Legislators are often not held accountable by 
learners or educators for providing adequate resources and policy 
guidance to the system. 

Capacity is weak. Adult basic education is struggling to create a national 
accountability system without a national service delivery system. It is 
difficult to have a management information system when there is no 
management system. 

Research reveals a fragmented and incomplete system with multiple 
funding sources and reporting formats, diverse institutions, competing 
objectives, and missing or unreliable performance data. In most states, 
staff are part-time, and volunteers continue to have an essential role in 
student services. Per-student funding is low, and most programs are not 
able to meet client needs for childcare or transportation. While data on 
performance are fragmentary, what there are suggest that most learners 
do not stay long, make some initial learning gains, but may not make long- 
term skill gains. 

The capacity to perform - to achieve desired goals - is linked with the 
capacity to be accountable - to document achievements and measure 
results. Some states are beginning to use program performance data 
successfully to improve program services. More reliable accountability 
data are collected when they are used at the program level to meet 
program needs. 

Measurement tools are not up to the task. Learning is at the heart of 
ABE, and its measurement is of particular concern to performance 
accountability. Adult education cannot be accountable to learners or to 
policymakers without the ability to track learning of individuals, to 
demonstrate what has been learned, to compare learning across 
programs, and to judge learning against external standards. 

Yet standardized tests, the most widely used tools for measuring learning, 
have been criticized both by researchers and practitioners because they 
do not demonstrate what has been learned. They are also incompatible 
with new research-based conceptions of literacy as social practices rather 
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than isolated skills. Some programs are using various "authentic 
assessment" tools, such as portfolios, but so far these cannot compare 
learning between learners and across programs. Without external 
standards or criteria, authentic assessment will not meet policy needs. 

What change is happening? 

New initiatives are addressing performance accountability. The problems 
of accountability are well recognized and are beginning to be addressed 
at national and state levels. 

Equipped for the Future: The National Institute for Literacy's Equipped 
for the Future (EFF) project is a broadrbased system reform effort that has 
actively sought input from a wide range of stakeholders. These include 
stakeholders outside the adult education system (policymakers, 
employers, and civic leaders) as well as within it (adult learners, teachers, 
program administrators, and researchers). EFF has developed and 
validated a set of four purposes for adult education and lifelong learning: 
to have access to information, to give voice to ideas and opinions, to solve 
problems, and to be able to continue learning. EFF has related these 
purposes to the three adult roles of worker, citizen, and family member. 
With 25 development partners across the country, EFF is now testing a 
set of common activities and defined skills which will form the basis for 
future development of standards and performance measures. When fully 
articulated and validated, these could provide a framework for 
performance as the basis for an accountability system. 

National Outcomes Reporting System: This project is funded by the 
U.S. Department of Education and has involved many state ABE 
directors. It is developing a common set of outcomes for adult basic 
education as a basis for data collection and reporting. However, the 
process has so far had limited input from the full range of stakeholders. 

Other state and national level initiatives: A number of initiatives at 
state and national levels are bringing stakeholder groups into closer 
relationships. A National Summit, proposed by the National Center for the 
Study of Adult Learning and Literacy (NCSALL), in partnership with NIFL 
and the Department of Education, is an example of one way to ensure 
stakeholder involvement in deliberations about the future of ABE at the 
national level. Local and state adult learner organizations are forming, 
increasing learner engagement around issues like state and federal 
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funding. A national adult learner organization, VALUE (Voices for Adult 
Literacy United for Education), has recently been formed and shows 
promise for building a strong and effective student voice. 

State capacity-building: Over the last few years, many states have been 
focusing on building capacity for program delivery. Some have worked 
specifically on building capacity for accountability as well. The cases of 
Arkansas, Connecticut, and Pennsylvania, reviewed in this policy paper, 
as well as other states like Massachusetts, Iowa, and California, all 
demonstrate the importance of long-term and systematic investment in 
training, information, and technical assistance. 

Lessons from the business world: High performance or "learning 
organizations" from the business world may prove useful models for adult 
education. These emphasize continuous improvement and learning in all 
parts of the organization, responsiveness to internal and external 
customers, participation in decisions, and shared responsibility at all 
levels. Traditional models of accountability are linear - quality control 
inspectors check widgets at the end of the production line to see if they 
meet specifications. Mutual accountability engages members of the 
organization in developing common vision, determining goals and 
customer expectations, and designing effective means of monitoring, 
producing, and improving. 

What next? A Framework for Action 

The policy paper recommends four principles which would enable the field 
to perform effectively and to be held accountable for performance. This 
framework for future policy builds on what has been learned about 
performance accountability in past experiences, and provides guidelines 
for future action. The principles are: 

1. Agree on performance. 

2. Build mutual accountability relationships. 

3. Develop capacity both to perform and be accountable. 

4. Create new tools to measure performance. 

1. Agree on performances 

Good performance - what needs to be measured - is not a technical 
question, but inevitably in the realm of values. The challenge is to come 
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to agreement on performance as a "big tent" which can include the full 
diversity of purposes. 

Experience from business and industry suggests that it is crucial that 
performance be defined neither too tightly nor too loosely. If performance 
is defined too tightly, a mis-match between system goals and individual 
goals may occur. For example, learners' purposes for entering literacy 
education may be to read to their children, but the system may only 
measure whether they get a GED. If performance is framed too loosely, 
no shared mission or common accountability measures can be developed. 
Getting it right requires a broad-based and inclusive process involving 
multiple stakeholders. 

Lessons from the literature and experience in education and other fields 
suggest: 

□ Don’t assume the question of what performance means can be 
skipped over or rushed. Without knowing what is important, 
measurement becomes an exercise in "gaming the numbers" to satisfy 
external demands, often with perverse results. 

□ Involve stakeholders and seek consensus. Without broad public 
debate it is difficult to frame performance goals which reflect the "big 
tent." 

□ Reflect newer understandings of literacy, and connect performance 
with real life. This is an opportunity for literacy research to connect 
with and support practice. 

□ Acknowledge multiple performances. Too narrow or tight a definition 
of goals will exclude learners and programs or force them to falsify 
their data. 



2. Develop mutual accountability relationships 

Reforming accountability in high performance terms requires a switch from 
one-way, top-down lines of accountability to a mutual web of 
accountability relationships. To participate, stakeholders need information 
and the ability or power to hold others accountable. This entails greater 
transparency in information, increased flows of information, and room at 
the table for groups who have not been there in the past. 

□ Bring the full range of stakeholder groups into the process - including 
teachers and learners who often have not been at the table. 
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□ Provide support for stakeholders who have least access to information 
and power, including adult learner organizations at national and state 
levels. 

□ Increase information flows among and between all stakeholders. 

□ Develop learning organizations at the program and state levels which 
would emphasize learning and continuous improvement, shared 
responsibility, and engagement in monitoring results. 

3. Build capacity to perform and be accountable 

State experiences with capacity-building reviewed for this policy paper 
indicate that the two kinds of capacity - to perform and to be accountable 
- are linked. By developing a learning organization approach in which 
there are continuous feedback loops, performance data can help 
programs improve performance. 

□ Build the capacity to perform. Key elements include increased 
resources, focusing resources on quality rather than quantity, staff 
development and training, technical support, use of performance data 
for continuous improvement. 

□ Build the capacity to be accountable. Key elements include 
accountability demands which are commensurate with resources and 
capacity, engagement of users in developing better measurement 
tools, staff training and support, timely information loops, rewards for 
improved performance. 

4. Create new tools to measure performance 

Accountability systems must meet the different information needs of 
different stakeholders. To do so, data users and data providers must be 
in communication so that the most appropriate measurement tools can be 
applied. 

Accountability systems commonly use several types of indicators to track 
performance over time. Input indicators provide information about the 
capacity of the system and its programs. Process indicators track 
participation in programs to see whether different educational approaches 
produce different results. Output indicators are short-term measures of 
results, and outcome indicators are long-term measures of outcomes and 
impacts. No single indicator can suffice to measure performance, 
especially of an enterprise as complex as adult basic education. 
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ABE invested a great deal of work into developing indicators of program 
quality, but much less on performance measures. New approaches and 
tools for measurement are needed which are linked to performance. 

□ Develop external standards or criteria against which individual student 
learning can be measured and through which program performance 
can be assessed. 

□ Develop performance assessment tools for measuring learning. 

These directly assess learners' performance in terms of literacy 
practices rather than the indirect approaches of standardized tests 
which "stand for" real-life practices, usually inadequately. Initiatives in 
performance assessment in countries such as Britain and Australia 
may provide useful models for measuring and assessing learning. 

□ Use the full range of potential of research, evaluation, and monitoring 
technologies to meet the needs of different stakeholders. These 
approaches to gathering, analyzing, and using information are based 
on different kinds of data and meet different purposes. Using them in 
appropriate ways, adult education can develop a dynamic system of 
information, analysis, and reporting. 

Next steps 

This framework of principles for action acknowledges that there are no 
quick answers. To put the principles into action requires consultation with 
the field and with stakeholders. It will need meetings and taskforces, and 
it will take time. It requires learning lessons from elsewhere when 
appropriate, building on current initiatives when they are under way, and 
creating new tools when none exist. Policymakers have the capacity to 
set the stage, harness resources, and create a common agenda. 
Commitment to high performance requires the contributions of many 
players. 
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INTRODUCTION 

This paper hopes to contribute to the public debate that must answer 
fundamental questions about accountability in adult education. To whom 
are adult educators accountable? For what? How shall we demonstrate 
performance? 

To be accountable is to be responsible to others for what we do. In recent 
years responsibility for achieving results in adult education has been an 
outgrowth of politicians (and taxpayers) demanding to see their “return on 
investment.” The focus has shifted from process (What does a good 
quality program look like?) to outcomes (What are the results of 
participation in programs?). Outcomes can no longer be taken for 
granted, but must be measured and proved. But just what these 
outcomes should be is not always clear or agreed, either within the field or 
outside it. 

State and national efforts have begun to construct systems to report on 
the results of adult basic education . 1 Some are designed to provide data 
for program improvement, some primarily to report to funders. These 
efforts include the National Institute for Literacy’s Performance 
Measurement and Reporting Improvement System (PMRIS) projects, the 
National Governors Association’s workforce development projects, the 
U.S. Department of Education-led program quality indicators initiative, and 
the National Outcomes Reporting System. Volunteer literacy 
organizations and some other states have their own performance 
accountability initiatives. 

To measure performance we have to know what success would look like, 
what the desired outcomes are - in fact, what it means to be literate. 
Currently, it seems that policymakers, teachers, learners, and the general 
public hold many and varied ideas about what the outcomes of adult basic 
education should be. Is the goal for individuals to acquire increased 
literacy skills, greater self confidence, to participate more in their 
community, make more effective parents, acquire better jobs? Is the 



' A note about terms here: adult literacy education and adult basic education are used 
interchangeably. They include basic literacy skills (reading and writing) and language 
skills (speaking and listening) as well as numeracy. The legislation which governs a great 
deal of this in the United States is the Adult Education Act, and for the sake of brevity the 
term “adult education” is sometimes used here in this sense. Nevertheless, the field of 
adult education as a whole is broader, addressing all the myriad learning experiences 
and opportunities encountered in adult life. 
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benefit for society having higher skilled workers, a more active citizenry, 
stronger families? Does literacy have value in itself, or only because of its 
social impacts? Is literacy as firmly associated with social impacts as 
popularly believed? There are also different understandings of literacy 
itself: as an autonomous skill or set of skills; as the ability to function in 
society; as diverse socio-cultural practices rooted in particular contexts. 
Without clarity and agreement on these fundamental questions, it is 
difficult to see how performance accountability systems can work. 

Getting clear about what we mean by “performance" is not just a technical 
question, but a question of value. At the heart of performance 
accountability is what we want adult basic education to be and to become. 
Only when we are clear about where we want to go can we create 
mechanisms to show how far we have come. The process of developing 
that clarity and agreement should start by engaging the field, and must 
reach beyond it to the broader public. 

This policy paper seeks to make a contribution to that debate by 
examining underlying assumptions, unpacking terms, and identifying ways 
forward in developing performance accountability in adult basic education. 
It is written not just for people who think of themselves as policymakers, 
but for a wider audience who are active and committed to adult basic 
education, and who want to play a role in shaping its future. Preparation 
for the policy paper included three main activities: 

1 . A review of literature from education and other fields. This was a 
means of identifying major issues to be addressed in the policy paper. 

It drew on materials developed by the federal government including 
the U.S. Department of Education, the National Governors 
Association, the National Institute for Literacy, and several states. It 
ranged further into recent research and theory about literacy, 
particularly New Literacy Studies, organizational management, and in 
particular, performance indicators and measurement in both the 
national and international arenas. 

2. A series of interviews with adult education researchers and leaders at 
state and national levels about their performance accountability efforts. 
These were conducted in March and April, 1997, in person and by 
phone. The 17 individuals included current or former state directors of 
adult education, people active in national policy, leaders of national 
volunteer literacy organizations, researchers with extensive experience 
in the field, and others experienced in state-level performance 
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management systems. The interviews were not intended to be 
representative of the field as a whole, but rather to draw on a 
range of experiences with performance accountability in order to 
identify major issues and questions. 

3. A discussion workshop to reflect on the emerging issues with a group 
of adult education practitioners and researchers. Participants were 
selected for their perspectives at national, state and program level. 
They were not necessarily experienced in performance accountability, 
but were very experienced in the field - in workplace literacy, family 
literacy, English for Speakers of Other Languages (ESOL), 
community-based and institution-based programs. 

The paper starts with the context, in terms of societal, governmental, and 
educational changes. In Chapter Two, some of the key terms are 
examined: accountability, performance accountability, and performance - 
with a look at conceptions of literacy itself, and the purpose of literacy 
education. In Chapter Three, the issue of capacity - both to perform and 
to be accountable - is discussed, along with specific issues around 
measurement. Finally, in Chapter Four, a scenario is sketched out for 
ways forward in terms of debate, action, and additional research. 
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